Target-Free Text-Guided Image Manipulation

We tackle the problem of target-free text-guided image manipulation, which requires one to modify input reference based on given text instruction, while no ground truth target is observed during training. To address this challenging task, we propose a Cyclic-Manipulation GAN (cManiGAN) in paper, able realize where and how edit regions interest. Specifically, editor cManiGAN learns identify complete image, cross-modal interpreter reasoner are deployed verify semantic correctness output instruction. While former utilizes factual/counterfactual description learning for authenticating semantics, latter predicts "undo" instruction provides pixel-level supervision training cManiGAN. With above operational cycle-consistency, our can be trained weakly supervised setting. conduct extensive experiments datasets CLEVR COCO datasets, effectiveness generalizability proposed method successfully verified. Project page: sites.google.com/view/wancyuanfan/projects/cmanigan.